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ABSTRACT 

Synthetic Vision Systems and Enhanced Flight Vision System (SVS/EFVS) technologies have the potential to provide 
additional margins of safety for aircrew performance and enable operational improvements for low visibility operations 
in the terminal area environment with equivalent efficiency as visual operations. To meet this potential, research is 
needed for effective technology development and implementation of regulatory and design guidance to support 
introduction and use of SVS/EFVS advanced cockpit vision technologies in Next Generation Air Transportation System 
(NextGen) operations. 

A fixed-base pilot-in-the-loop simulation test was conducted at NASA Langley Research Center that evaluated the use 
of SVS/EFVS in NextGen low visibility ground (taxi) operations and approach/landing operations. Twelve crews flew 
approach and landing operations in a simulated NextGen Chicago O’Hare environment. Various scenarios tested the 
potential for EFVS for operations in visibility as low as 1000 ft runway visibility range (RVR) and SVS to enable lower 
decision heights (DH) than can currently be flown today. Expanding the EFVS visual segment from DH to the runway 
in visibilities as low as 1000 RVR appears to be viable as touchdown performance was excellent without any workload 
penalties noted for the EFVS concept tested. A lower DH to 150 ft and/or possibly reduced visibility minima by virtue of 
SVS equipage appears to be viable when implemented on a Head-Up Display, but the landing data suggests further study 
for head-down implementations. 

Keywords: Synthetic Vision, Enhanced Vision, Enhanced Flight Vision System, Head-Up Display, Aviation Safety, 
Flight Deck Systems, All Weather Operations 


1. INTRODUCTION 

The U.S. air transportation system is undergoing a transformation to accommodate the movement of large numbers of 
people and goods in a safe, efficient, and reliable manner. 1 One of the key capabilities envisioned to achieve this Next 
Generation Air Transportation System (NextGen) is the concept of equivalent visual operations (EVO). EVO is the 
capability to achieve the safety of current-day Visual Flight Rules (VFR) operations and maintain the operational tempos 
of VFR irrespective of the weather and visibility conditions. 

One research challenge for EVO is the definition of required equipage on the aircraft and on the airport. With today’s 
equipment and regulations, significant investment is required in on-board equipment for navigation, surveillance, and 
flight control and on the airport for precision guidance systems and approach lighting systems for “all-weather” landing 
capability. 2 The levels of equipment redundancy, capability, maintenance, performance and crew training dramatically 
increase as landing visibility minima decrease. Synthetic Vision Systems and Enhanced Flight Vision Systems 
(SVS/EFVS) offer a means of providing EVO capability without significant airport infrastructure investment or training. 

NASA Langley Research Center (NASA LaRC) and the Department of Transportation/Federal Aviation Administration 
(DOT/FAA) are jointly conducting collaborative research to ensure effective technology development and 
implementation of regulatory and design guidance to support introduction and use of SVS/EFVS advanced cockpit 
vision technologies in NextGen operations. These technologies have the potential to provide an additional margin of 
safety and aircrew performance to enable the implementation of operational improvements for low visibility surface, 
arrival, and departure operations in the terminal environment with equivalent efficiency as visual operations. This work 



builds from and extends the current operational use and certification of existing SVS/EFVS technologies to serve as 
building blocks toward all-weather, low visibility operations for NextGen. 


2. BACKGROUND 

S VS is a computer-generated image of the external scene topography, generated from aircraft attitude, high-precision 
navigation, and data of the terrain, obstacles, cultural features, and other required flight information. EFVS is an 
electronic means to provide a display (typically on a head-up display, or HUD) of the external scene by use of an 
imaging sensor, such as a Forward-Looking InfraRed (FLIR) or millimeter wave radar. Both SVS and EFVS are 
“vision-based” technologies intended to create, supplement, or enhance the natural vision of the pilot. 

NASA and others have developed and shown SVS technologies that provide significant improvements in terrain 
awareness and reductions for the potential of Controlled-Flight-Into-Terrain incidents/accidents, 3 " 5 improvements in 
flight technical error to meet Required Navigation Performance criteria, 6,7 and improvements in situation awareness (SA) 
without concomitant increases in workload compared to current generation cockpit technologies. 8 " 12 As such, SVS, often 
displayed on a Head-Down Display (HDD), is emerging as “standard equipage” for Part 23 and Part 25 business and 
general aviation aircraft flight decks even though, to date, no “operational credit” is obtained by SVS equipage. 

EFVS capability on a HUD using FLIR sensor technology has also carved out a significant share of the business aircraft 
market and is growing in Part 121 operations. EFVS provides many of the same operational benefits as SVS technology, 
but it uses a direct view of the external environment, independent of the aircraft navigation solution or database. These 
differences, in part, enable operational credit by use of an approved EFVS. In 2004, Chapter 14 of the Code of Federal 
Regulations (CFR) Section §91.175 was amended such that operators conducting straight-in instrument approach 
procedures (in other than Category II or Category III operations) may now operate below the published Decision 
Altitude (DA), Decision Height (DH) or Minimum Descent Altitude (MDA) when using an approved EFVS shown on 
the pilot’s HUD. 

Synthetic and Enhanced Vision Systems (SEVS) technologies, such as SVS/EFVS in combination with HDD/HUD, 
form the basis for an electronic display of visual flight references (terrain, obstacles, and operations-critical navigational 
and situational references) on electronic cockpit display(s) for the flight crew, integrated with conformal symbology 
providing important situation, guidance, and/or command information as necessary and/or appropriate to enable all 
weather approach and landing operations. The primary reference for maneuvering the airplane is based on what the pilot 
sees through the SEVS, in lieu of or supplemental to the pilot’s natural vision. 

The key concept under the revisions to §91.175 is that an EFVS can be used in lieu of the required natural vision from 
the DA/DH/MDA to 100 feet (ft) height above the touchdown zone elevation (TDZE). Minimum aviation system 
performance standards are now available (RTCA DO-315 13 ) and an FAA Advisory Circular (AC 90-106 14 ) provides 
guidance for design and certification criteria of EFVS for operational credit. DO-315 also provides performance 
standards for SVS but with no additional operational credit. In other words, installing SVS does not change the 
airplane’s existing operational capability or certification basis. 

The emerging challenge for NextGen - and the subject of NASA research - is to develop performance-based standards 
for SEVS technologies that create EVO and beyond. The first part of this challenge is the development of performance- 
based standards expanding the current operational approvals of SVS/EFVS. 

The design and use of SEVS technology is integrally tied to the operating rules for landing and take-off under Instrument 
Flight Rules (IFR) (14 CFR §91.175). For EFVS operations, §91.175 was modified such that, to operate below the 
DA/DH or MDA, certain visual references must be seen using the EFVS to continue the descent below the published 
DA/DH or MDA (see 14 CFR §91.175). No lower than 100 ft height above the TDZE, the visual references that 
positively identify the runway of intended landing must be distinctly visible and identifiable using natural vision. 

The FAA has started a rulemaking project to expand operational credit for EFVS beyond what is currently authorized by 
the regulations (under 14 CFR §91.175). DO-315A 15 was drafted to establish performance standards in concert with this 
rulemaking project. Minimum performance standards are now published for EFVS operations through the approach to 
touchdown in visibility as low as 1000 ft runway visual range (RVR) by sole use of an approved EFVS in lieu of natural 
vision. Simply stated, the visual segment of the approach can now be accomplished by using either enhanced flight 
visibility or natural vision. Past NASA research supports the viability of this expanded EFVS visual segment that 



showed that using an EFVS through touchdown resulted in excellent localizer tracking performance and an improvement 
in glideslope tracking performance. 16 

The joint RTCA SC-213/EUROCAE WG 79 committee is also working on minimum performance standards for possible 
operational credit for SVS (under DO-315B, in RTCA Final Review and Comment process). Unlike EFVS, the possible 
path for operational credit is not through revision of 14 CFR §91.175, but is based on FAA Order 8400.13 (“Procedures 
for the Evaluation and Approval of Facilities for Special Authorization Category I Operations and All Category II and III 
Operations”). Specifically, DO-315B establishes performance standards for a SVS enabling lower than standard 
Category I minima or a reduction in the required minimum visibility. These proposed DO-3 1 5B performance standards 
for SVS operational credit do not require the use of a HUD. 

A fixed-base experiment was conducted to evaluate selected elements of the proposed performance standards for 
expanded EFVS and SVS operational credits. Specifically, the high-level objectives of this simulation test were to: 

• Evaluate the operational feasibility, pilot workload, and pilot acceptability of conducting a straight-in 
instrument approach procedure with published vertical guidance using EFVS (i.e., FLIR imagery on a HUD) 
for the approach, landing, roll-out and turn-off in weather and visibility as low as 1000 ft RVR. 

• Evaluate the operational feasibility, pilot workload, and pilot acceptability of conducting a straight-in 
instrument approach to a 150 ft DA/DH procedure with published vertical guidance using SVS (displayed either 
head-down or head-up) and to transition to natural out-the-window (OTW) visual conditions for landing in 
weather and visibility as low as 1400 ft RVR. 

In addition, the following test objectives were evaluated in this experiment, but are not reported in this paper: 

• The influence of selected DO-315 minimum performance standards items for EFVS (e.g. EFVS failure at 50 ft 
above field level (AFL) in varying RVR conditions) and SVS (e.g., un-alerted navigational position errors at 
Wide Area Augmentation System horizontal alert limit and vertical alert limit values) 

• Evaluation of time required, accuracy, and pilot workload associated with recognizing and reacting to potential 
ground collisions or conflicts with other aircraft, vehicles and obstructions across a range of visibility and 
lighting conditions using an EFVS and SVS 

- Using single and dual pilot operations. 

- With and without the cockpit display of traffic information (CDTI). 

Further, eye gaze data were collected to quantify the pilot’s visual attention when flying the instrument segment, the 
transition from instrument to visual, and during the visual segment. Analyses of these data are ongoing and not reported 
here. 


3. METHODOLOGY 


3.1 Subjects 

Twenty- four pilots served as participants for the research, representing twelve flight crews. Ten crews flew for major 
U.S. air carriers and were paired by airline to ensure crew coordination and cohesion with regard to terminal and surface 
operational procedures. The other crews were business aircraft operators, flying Gulfstream G-V or G450 aircraft 
equipped with EFVS and SVS. The Captains had an average of over 14,661 flight hours with 21 years of commercial 
flying. The First Officers had an average of over 10,648 flight hours with 14 years of commercial experience. The 
Captains were recruited on the basis of HUD experience (at least 100 hours), with preference given to pilots with 
Enhanced Vision (EV)/EFVS experience. All pilots were required to hold an Airline Transport Pilot rating. 

The Captain was the designated pilot- flying (PF) throughout all the trials and the First Officer served as the pilot-not- 
flying (PNF) or monitoring pilot. 

3.2 Simulation Facility 

This research was conducted in the Research Flight Deck (RFD) simulator at NASA FaRC (Figure 1). The RFD is 
configured with four 10.5-inch Vertical (V) by 13.25-inch Horizontal (H), 1280x1024 pixel resolution color displays, 



tiled across the instrument panel. Also, the RFD includes a HUD on the left side of the cab, Mode Control Panel, Flight 
Management System (FMS), and two Electronic Flight Bags (EFBs) (Figure 1). 

The full-mission RFD simulates a Boeing B-757-200 aircraft, albeit controlled through sidestick inceptors. The cab is 
populated with flight instrumentation and pilot controls, including the overhead subsystem panels, to replicate the B-757. 

A collimated OTW scene is produced by an Evans and Sutherland Image Generator graphics system providing 
approximately 200°H by 40°V field-of-view (FOV) at 26 pixels per degree. 

The sidestick inceptor force gradients and deflection characteristics mimic the Airbus 320 aircraft. 17 A parabolic 
shaping gearing was used between the normalized stick deflection of the sidesticks and the elevator and aileron 
commands of the B-757 aircraft. The parabolic shaping provided acceptable handling characteristics for the approach, 
landing, and departure tasks. The pilot and co-pilot inceptors are directly linked as if mechanically connected. 

The auto-throttle system backdrives the throttle handles to directly reflect the power setting commanded to the engines. 
Take-off, go-around (TOGA) buttons and autothrottle disconnect buttons are placed on the throttle handles. 



Figure 1. Research Flight Deck Simulator with HUD, Head-down Instrument Panel and EFB 

3.2.1 Simulator Database 

Operations were simulated at Chicago O’ Hare International Airport (FAA identifier: ORD). The simulation was built 
around FAA source data for ORD, valid from 1 1 March 2010 to 8 April 2010. These data were used to develop all flight 
plans, scenarios, approach paths, and OTW, synthetic vision (SV) and EV databases. 

Day simulations were flown, with the weather tailored to create the desired visibility conditions. 

Approaches were flown only to runways with Medium intensity Approach Lighting System with Runway alignment 
indicator lights (MALSR) installed. Testing included an experimental variation of touchdown zone and centerline 
(TDZ/CL) lights (on and off), where operations with TDZ/CL lights were conducted on ORD Runway 9R; otherwise, 
ORD Runways 4R, 22L, or 22R were used. All runways included high intensity runway lights and serviceable 
centerline and surface markings. Airport lighting was drawn using calligraphics. 

3.2.2 Audio Effects 

Altitude call-outs were played over the flight deck speakers. The automatic altitude calls-out started at “500 feet” when 
the aircraft was 500 ft above the TDZE. The “approaching minimums” and “minimums” call-outs were generated at 100 
ft above and at the planned DA/DH for a given run. 



Flare “prompts” in the form of additional altitude call-outs were used on all runs (“100,” “50,” “40,” “30,” “20,” and 
“10” at the corresponding radar altitudes in feet). 

3.2.3 Head-Down Displays 

Figure 1 shows the simulator’s four main instrument panel displays: a) PF left display, including primary flight display 
(PFD); b) PF right display including navigation display (ND); c) PNF left display, including ND; and, d) PNF right 
display, including PFD. The format and content of these displays were varied experimentally as described below. 

3.2.4 Head-Up Display 

The RFD is equipped with a Rockwell-Collins HGS-4000 HUD. The HUD is collimated and subtends approximately 
26° H by 21° V FOV. The HUD projects the imagery from a Cathode Ray Tube source in a stroke-and-raster format 
(Figure 2). The raster input to the HUD was either a SV or EV source imagery in an RS-343 format. The stroke 
symbology format was a modified version of the HGS Primary mode format. The stroke symbology included a runway 
outline (edgelines) which was conformally drawn until 50 ft AFL, a flight path angle reference cue and a flight-path 
referenced guidance cue. The guidance cue was driven by the B-757 flight director. In addition, a HUD flare cue, 
consisting of a flare “prompt,” was provided where, at 50 ft AFL, two “plus” signs flashed above the flight path marker. 



Figure 2. EFVS HUD (left) and SYS HUD (right) Symbology Format 


The PF had independent controls to adjust the stroke symbology brightness and the raster imagery brightness and 
contrast. The PF also had a declutter control, implemented as a four-button castle switch on the pilot’s sidestick. The 
four “declutter” states available to the PF were: 1) Declutter All (no symbology or imagery); 2) Symbology (Stroke) 
Toggle on/off; 3) Imagery (Raster) Toggle on/off; and 4 Display All (both symbology and imagery). 

The HUD was stowed when not being used to avoid any confounding from the HUD being in place during “non-HUD” 
runs. 

3.3 SV Simulations 

A SV database was developed by NASA starting from the OTW database, generally following the standards from DO- 
315B (draft). The database used a 1 arc-second Digital Elevation Model (DEM) of a 110.25 nm (East-West) by 145.6 
(North-South) nm area centered around ORD. The DEM was draped with an elevation-based coloration texturing. 

Each ORD runway was modeled as an asphalt-colored polygon using the threshold data and runway widths. Threshold 
lines, edgelines, and runway numbers were added. 

The intended landing runway (as selected through the FMS prior to run initiation) was denoted on the primary display 
concept being evaluated, either as a conformal magenta outline on the head-down PFD depiction, or an 8000 x 200 ft 
outline (shown as edgelines) on the HUD. 

Because the test was confined to low altitude approach, landing, and surface operations at ORD, obstacles would not 
create a significant visual cue. Therefore, obstacles were not included or marked in the SV depiction. 



The SVS-PFD symbology mirrored the HUD using conformal depictions for the flight path marker, single cue flight 
path-referenced guidance symbology, and flight path angle reference cue. Other required Primary Flight Reference 
information was also drawn (e.g., airspeed, altitude, and raw data deviations). 

Several SV-specific items are called out to note: 

• When drawn on the HUD, the SV database terrain texturing and coloration was slightly changed to improve its 
visual perception primarily by specific coloration for conversion into a gray-scale format. 

• The SV depiction was always drawn in a heading-up format. Any crosswind was evident by conformal lateral 
positioning of the flight path marker. However, the flight path marker and guidance cue were limited and 
displayed as ghosted representations if their conformal positions exceeded pre-determined values. 

3.4 EV Simulation 

The EV real-time simulation is created by the Evans and Sutherland EPX physics-based sensor simulation. The ORD 
database was instantiated with material code properties. From this database, an IR sensor simulation, interacting with 
this material-coded database and the simulated weather conditions, created the desired test experimental conditions. 

The EV simulation mimicked the performance of a short- wave/mid- wave FLIR, using a ~ 1.0 to 5.0 micron wavelength 
detector. The nominal enhanced visibility was approximately 2400 ft for this experiment. 

The eye point reference for the EV simulation was placed 5 ft below the pilot design eye reference point, but otherwise 
properly boresighted (i.e., angular alignment) to the aircraft. In the B-757, the pilot is approximately 20 ft above the 
ground during surface operations. This EV eye point reference/parallax error generates 2.5 mrad error to a point located 
2000 ft away - approximately half of the accuracy budget of the EFVS per current DO-315 accuracy requirements. 13 

3.5 Navigational Performance Variations 

Variations in navigational accuracy were simulated on each run, bounding ±12 ft vertical and ±12 ft horizontal 
deviations from the true position. This effect was added for realism in positioning system accuracy. The selected 
inaccuracies were randomly varied across each subject’s test matrix and were held constant during a run. 

3.6 Crew Display Concepts 

Two head-down flight display concepts and three head-up flight display concepts were evaluated by the crews while 
flying approaches, landings, and surface operations to Runways 4R, 9R, 22L or 22R at ORD. 

3.6.1 Head-Down Flight Display Concepts 

The two HDD concepts (referred to as the Baseline HDD and SVS HDD) are shown in Figure 3, differing from each 
other only in the absence or presence of SVS on the PFD. The HUD was stowed during HDD evaluations. 

The SVS on the PFD portrayed a 33° V x 44° H field-of-regard. Assuming a 25 inch distance from the Design Eye 
Reference Point to the display, the SVS concept had a minification factor of approximately 2.1 for the PF. The PF left 
display also had a datalink message area and Horizontal Situation Indicator (HSI). The PNF right display showed a 
quad- view of flight information: a PFD (upper left); HSI (lower left); FLIR repeater or blank area (upper right); and 
datalink message area (lower right). The upper right area of the quad-display was blank for baseline and SVS HDD 
evaluations. 

The PF (right display) and PNF (left display) NDs always showed flight traffic and navigational information in the 
airborne mode (see Airborne Modes on Figure 3). The PF and PNF NDs transitioned to a moving map mode when on 
the ground and groundspeed less than 100 knots (see Surface Modes on Figure 3). The PNF ND included a runway inset 
view in both airborne and moving map modes. 

The surface moving map and the runway inset views always showed their respective images of the airport and intended 
landing runway, but traffic icons (i.e., cockpit display of traffic information, or CDTI) were only drawn when being 
experimentally evaluated. These experimental variations (e.g., the presence and absence of CDTI) were conducted to 
test the time required, accurate identification, and pilot workload associated with recognizing and reacting to potential 
ground collisions or conflicts with other aircraft. 



Airborne Modes 




Figure 3. Head-Down Display (HDD) Concepts: Baseline HDD on top and SVS HDD on bottom 

3.6.2 Head-Up Flight Display Concepts 

The three HUD concepts (referred to as the Baseline HUD, SVS HUD and EFVS) were tested, differing from each other 
only in the absence or presence of imagery (SV or FLIR) on the PF HUD. 

The Baseline HDD (i.e., no SVS) concept was displayed during these runs. The PNF (head-down) right display showed 
FLIR imagery in the upper right corner of the quad-view for EFVS runs and was blank (black in color) for Baseline and 
SVS HUD runs. Figure 2 (in the left-side view) shows the EFVS display concept with symbology and FLIR imagery. 

3.7 Evaluation Task 

The evaluation task was a straight-in Instrument Landing System (ILS) approach with a 3 degree descent angle to one of 
4 ORD runways (4R, 9R, 22R, or 22L). For each run, the approach started 3 nautical miles (nm) from the runway 
threshold. The weather consisted of low to moderate winds with either 10 knot headwind, 10 knot tailwind, 7.5 knot 
crosswind, or 15 knot crosswind, light turbulence (root-mean- square (rms) of 1 ft/sec), and varying OTW visibility 
levels (1800 ft, 1400 ft, or 1000 ft RVR). When used experimentally, the FLIR visibility was 2400 RVR. The PF hand- 
flew the approach from the left seat with the auto-throttle set to “speed-hold” at the approach speed of 130 knots 
indicated airspeed. The auto-throttle automatically reduced to idle thrust at 35 ft above ground level (AGL) for landing. 
The run was terminated once the PF completed the landing, roll-out and turnoff or upon go-around initiation. The 
aircraft was configured to land prior to each run (landing gear down and flaps 30 degrees). 

The PFs were instructed to fly the aircraft as if there were passengers aboard, track the approach path, and land within 
the touchdown zone with an acceptable sink rate. After landing, they were to capture the centerline and exit at the 
expected taxiway at a speed to 5 to 15 knots at the 90 degree exits or 30 knots at the high-speed exits. They were also 
instructed to initiate a go-around if the landing was not safe or there were any safety concerns during the approach. 

3.8 Crew Procedures 

The crew was trained in ‘monitored approach’ crew procedures. The PF flew the approach using the HUD or HDD as 
the primary flight reference. The PNF monitored using the available HDD information, including a FLIR repeater (when 
EFVS was flown), and the OTW scene and assisted the PF as appropriate and necessary. There was no transfer of 
control from the PF to PNF (or vice versa). 








To facilitate comparison between baseline conditions (i.e., no SVS or EFVS installed) and SVS/EFVS configurations, 
crew procedures were standardized and trained. 

3.8.1 Baseline and SVS Procedures 

The procedures for the baseline (i.e., no SVS or EFVS installed) and SVS configurations were identical and followed 
normal crew instrument approach procedures. One of the intended functions of the SVS is to improve the pilot’s ability 
to conduct the instrument portion of the approach - not to enable descent below the published DA/DH. Therefore, no 
change in crew procedures would be expected. 

3.8.2 EFVS Procedures 

The EFVS procedures were built around common practice in current EFVS operations and FAA requirements (14 CFR 
§91.175 (1)) but extended to emphasize that to descend below the DA/DH and to descend below 100 ft height above the 
TDZE depended upon the PF being able to recognize and identify the required visual references, using EFVS. 

3.9 Experiment Matrix 

The primary experiment test matrix aligned three visibility conditions against five SEVS configurations (combinations 
of vision systems and displays) as shown in Table 1. 


Table 1. Experiment Matrix 


Vision 

Display System 

1800 RVR 

Visibility 
1400 RVR 

1000 RVR 

HUD None 


V 


SVS 


V 

V 

EFVS 


V 

V 

HDD None 

Z* 

V 


SVS 

V 

V 



* 200 ft DA; All others 150 ft DH 


Wind variations were balanced across the experiment matrix for each crew/pilot to evenly distribute the conditions 
across the configurations. Thus, wind effects were tested but not in a ‘within subjects’ variation. It was assumed that 
left and right crosswinds could be interchanged without affecting any experimental results. 

3.10 Measures 

During each approach and landing run, path error, pilot control inputs, and touchdown performance (sink rate and speed 
at touchdown, distance fore or aft of touchdown zone, and distance left or right of centerline) were measured for 
analysis. During taxi operations, centerline tracking and taxi speed were measured. 

After each run, pilots completed the AFFTC Workload Estimate Scale. 18 After data collection was completed, pilots 
were administered two paired comparison tests: the Situation Awareness - Subjective Workload Dominance (SA- 
SWORD) 20 technique and one on Traffic Awareness evaluating CDTI formats tested. 

3.11 Test Conduct 

The subjects were given a one-hour briefing describing the experiment, HUD and HDD concepts, crew procedures, and 
evaluation tasks. The test purpose was described to the test subjects as “evaluating the potential use of EFVS and SVS 
for reduced landing weather minima and the influence of CDTI for NextGen operations.” 

After the briefing, a 1.5 hour training session in the RFD was conducted to familiarize the subjects with the aircraft 
handling qualities, display symbologies, pilot procedures, and controls. In particular, in-simulator training highlighted 
the crew procedures for EFVS and SVS operations and landing performance. Fanding performance was planned as one 
of the performance parameters used to assess the efficacy of the SEVS experimental variations. However, none of the 
pilots were familiar with the handling characteristics of the RFD simulator (a sidestick-equipped B-757). To 
accommodate this disparity, each PF was trained to an acceptable standard of approach and landing performance. 




In Table 2, touchdown performance criteria are shown. After each training run, a landing performance “scorecard” 
against these criteria was displayed for feedback. The pilots were asked to meet the desired performance criteria. 
Training concluded once the pilots demonstrated repeatable desired landing performance, albeit with an occasional 
adequate performance score. If the acceptable performance criteria were met, they landed within the touchdown zone 
with acceptable sink rates. 

The training was flown in varying OTW visibility from visual conditions down to 1000 ft RVR. Similarly, EFVS 
visibility ranged from unlimited down to 1 000 ft. The training emphasized that they must always remain safe and if they 
felt unsafe conditions exist, the necessary precautions, including a go-around, should be executed immediately. 

Table 2. Touchdown Performance Scorecard 


Performance Value 

Desired 

Adequate 

Not Adequate 

Lateral Distance from 
Centerline 

Within +/- 27 ft 

Between +27 and +58 ft or 
Between -27 and -58 ft 

> +/-5S ft 

Longitudinal Distance from 
Threshold 

Between 750 to 2250 ft 

Between 200 & 750 ft or 
Between 2250 & 2700 ft 

< 200 or >2700 ft 

Sink rate 

Between 0 to 6 ft/sec 

Between 6 to 10 ft/sec 

>10 ft/sec 

Airspeed (kts) 

Between V ref -5 to V re f+5 

Between V ref -5 to V re f-15 

< V ref -15 or > V ref +5 


V re f + 5 is the approach speed in table above 


4. RESULTS 


4.1 Landing Performance 

Touchdown statistics were used to evaluate how effectively the pilots could land with the different SEVS display 
concepts and how well the crews met the touchdown performance criteria (Table 2) to which they had been trained. 

SEVS landing performance was evaluated by using a fixed 1400 ft RVR/150 ft DH and conducting ANOVA (Analysis 
of Variance) analyses for the factors of display concept (Baseline HUD, SVS HUD, EFVS, Baseline HDD, and SVS 
HDD) and TDZ/CL lights (on, off) on touchdown longitudinal position, lateral position, and sink rate. EFVS-specific 
landing performance was evaluated by conducting ANOVAs for OTW visibility (1000 ft RVR, 1400 ft RVR) and 
TDZ/CL lights (on, off) on these same touchdown measures. Independent ANOVAs were conducted for SVS HUD and 
SVS HDD landing performance for the factors of OTW visibility and TDZ/CL lights. 

Finally, the ability to meet FAA AC 120-28D 19 performance-based landing standards were assessed across the display 
types and TDZ/CL light configurations tested. 

4.1.1 Touchdown Statistics 

In Table 3, the touchdown (T/D) data (mean, standard deviation, minimum value, and maximum value) are shown, 
broken out by display concept and TDZ/CL light configuration, for the fixed 1400 RVR OTW and 150 ft DH testing 
condition. Also provided in Table 3 are the number of runs that resulted in a TOGA and the total number of runs for the 
combinations of display concept and TDZ/CL light configurations tested. 

The data show that: 

• The decision by the pilots to safely continue the descent below the DH (150 ft) to landing depended upon the 
SEVS concept and the location of the information (head-up or head-down). 

o 92% (22 out of 24 runs) of the Baseline HUD runs were flown to a landing, with no apparent influence 
of the TDZ/CL lighting condition. 

o 100% (24 out of 24 runs) of the EFVS HUD runs were flown to a landing (i.e., no TOGAs), 
irrespective of the TDZ/CL lights. 

o 83% (20 out of 24 runs) of the SVS HUD approaches were flown to landing, with no apparent 
influence of the TDZ/CL lighting condition. 




o 


For the head-down condition, 83% (20 out of 24 runs) of the SVS HDD approaches were flown to 
landing, with more TOGAs flown with the TDZ/CL lighting condition. 

o Conversely, without SVS, crews successfully completed a landing only 50% of the time when flying 
the Baseline HDD concept to runways without TDZ/CL lights compared to 92% of the time (11 out of 
12 runs) to a runway with TDZ/CL lights. 

• The mean lateral position from centerline was within the “desired” T/D scorecard performance level (within ± 
27 ft) for all display concepts at 1400 RVR, irrespective of the TDZ/CL lights. 

• The mean T/D sink rate was only within the “adequate” performance level (between 6 to 10 ft/sec) for all 
display concepts at 1400 RVR, irrespective of the TDZ/CL lights. 

• The mean longitudinal T/D position performance varied by SEVS condition and the presence/absence of 
TDZ/CL lights: 

o The mean longitudinal T/D position performance was within the “desired” performance level (between 
750 to 2250 ft from threshold) for all EFVS HUD runs, irrespective of the TDZ/CL lights. 

o The mean longitudinal T/D position performance was within the “desired” performance level (between 
750 to 2250 ft from threshold) for all HUD SVS runs, irrespective of the TDZ/CL lights. 

o Although 92% of the Baseline HUD runs were flown to a landing, irrespective of the TDZ and CL 
lighting condition, only those flown to a runway with TDZ/CL lights met “desired” performance 
levels. The ones flown to runways without TDZ/CL lights only met “adequate” performance levels 
(between 200 & 750 ft or between 2250 and 2700 ft from threshold) for longitudinal T/D position. 

o The mean longitudinal T/D position performance was within the “desired” performance level (between 
750 to 2250 ft from threshold) for SVS HDD runs only with TDZ and CL lights. The ones flown to 
runways without TDZ/CL lights only met “adequate” performance levels for longitudinal T/D 
position. 

o All Baseline HDD runs met the “desired” level of performance for longitudinal T/D position, 
irrespective of the TDZ and CL lights. However, the presence of TDZ/CL lights had a profound effect 
on whether a landing was attempted - only 50% of approaches to a runway without TDZ/CL lights 
were flown to landing compared to 92% of the time to a runway with TDZ/CL lights. 

In Table 4, the touchdown data are shown for the 1000 ft RVR HUD runs and 1800 ft RVR SVS HDD runs, broken out 
by TDZ/CL light configuration. Note that one landing for the SVS HDD concept, 1800 RVR/150 ft DH, with TDZ/CL 
lights landed 50 ft short of the runway threshold (shown in red, bold font in Table 4 for emphasis). These data are 
included in the statistical analysis but the significance of this event cannot be ignored. 

Similar to the 1400 ft RVR values, Table 4 shows that the mean values for the lateral T/D distance met the “desired” 
levels for the 1000 RVR HUD runs and 1800 RVR HDD configurations. The mean values for the T/D sink rate only 
met the “adequate” levels for the 1000 RVR HUD runs and 1800 RVR HDD configurations. Also, like the 1400 ft RVR 
EFVS runs, there were no go-arounds for any of the 1000 ft RVR EFVS HUD approaches and all met the “desired” 
longitudinal T/D performance criteria. 

The data show the influence of SVS/EFVS on the pilots’ decision to safely continue the descent below the DH (150 ft) to 
landing. For the EFVS HUD runs, all approaches were flown to a landing even as the OTW visibility was reduced to 
1000 ft RVR. For the SVS HUD concepts, 71% (17 out of 24 runs) of the 1000 ft RVR runs were flown to touchdown, 
irrespective of the TDZ and CL lights. For the SVS HDD concept, 96% (23 of the 24 runs) of the 1800 ft RVR runs 
were flown to landing. 

Across almost every case , sink rates outside of the “acceptable” performance criteria were evident (as seen in the max 
sink rate values in Tables 3 and 4). 



Table 3. Touchdown Statistics for 1400 RVR and 150 ft DH Runs 
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Table 4. Touchdown Statistics for 1000 RVR and 1800 RVR Runs 
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4.1.2 ANOVA Analyses on Touchdown performance 

The touchdown data were analyzed for statistical significant differences. 

Fixed 1400 RVR QTW/150 ft DH runs : ANOVA analyses revealed that the presence of TDZ/CL lights had a highly 
significant effect (F(l,93)=9.57, p=0.003) on longitudinal distance from threshold (mean=1046 ft, standard deviation, 
g= 439 ft) compared to their absence (mean=803 ft, g= 356 ft). Display concept and the interaction between display 
concept and TDZ/CL lights were not significant (p>0.05) for longitudinal T/D distance (mean=928 ft, g= 417 ft). 

There were no significant (p>0.05) effects for the main factors (display concept, TDZ/CL lights) or their interaction in 
the lateral T/D position from centerline (mean= -0.18 ft, o=17.5 ft) or sink rate (mean= -7.3 ft/sec, g= 2.8 ft/sec). 

EFYS runs: No significant (p>0.05) OTW visibility level effects, TDZ/CL light effects, or second-order interaction 
effects were found on longitudinal T/D position (mean=984 ft, g= 381 ft), lateral T/D position (mean= -0.49 ft, o=14.5 
ft), or T/D sink rate (mean= -7.2 ft/sec, g= 2.9 ft/sec) for the EFVS display concepts. 

SYS HUD runs: ANOVA analyses revealed significant OTW visibility differences (F(l,33)=4.43, p =0.043) on 

longitudinal distance from threshold between the 1400 ft RVR level (mean=1012 ft, a=406 ft) and the 1000 ft RVR 
level (mean=757 ft, g= 347 ft). TDZ/CL lights and the interaction between OTW visibility and TDZ/CL lights were not 
significant (p>0.05) for this measure (mean=895 ft, g= 397 ft). 

There were also no significant (p>0.05) effects for the main factors (OTW visibility, TDZ/CL lights) or their interaction 
in the lateral T/D position from centerline (mean= 0.25 ft, ct= 17.0 ft) or sink rate (mean= -7.0 ft/sec, g= 2.7 ft/sec). 

SVS HDD runs : There were no significant (p>0.05) effects for the main factors (OTW visibility, TDZ/CL lights) or 
their interaction in the longitudinal T/D position (mean=860 ft, a=428 ft), lateral T/D position from centerline (mean= - 
0.18 ft, a=17.5 ft) or T/D sink rate (mean= -7.3 ft/sec, g= 2.8 ft/sec) measures. 

4.1.3 Objective Landing Standards Analysis 

Existing AC120-28D 19 (Appendix 3) performance-based “auto-land” standards for T/D longitudinal position, lateral 
position from centerline, and sink rate were applied in the objective landing data analysis. Specifically, the standards 
require no longitudinal touchdown earlier than a point on the runway 200 ft from the threshold or beyond 2700 ft from 
the threshold, no lateral touchdown with the outboard landing gear more than 70 ft from the runway centerline, and no 
touchdown sinkrate greater than -10 feet/second to a probability of 1 x 10" 6 . These standards pertain to the general 
concept of low- visibility approach and landings, but were not written specifically for operations with advanced vision 
systems such as SVS/EFVS. 

This experiment used an aim point located 1000 ft from runway threshold. For the simulated 757 aircraft, the outboard 
landing gear would be 70 ft from the centerline when the fuselage is at 58 ft lateral deviation from centerline, assuming 
no crab angle at touchdown. 

In Figure 4, the touchdown data are shown, broken out by display concept and TDZ/CL light configurations for the 1400 
RVR OTW and 150 ft DH crew runs. Included on this plot is the touchdown aim point (0 ft lateral distance, 1000 ft 
longitudinal distance) indicated by the axes origin. 

Visual inspection of the data shows that 1) all display concepts were within the lateral deviation from centerline 
touchdown criteria (within ±58 ft lateral distance) and 2) all the HUD runs (i.e., Baseline, SVS and EFVS) were within 
the longitudinal position touchdown criteria (between 200 ft to 2700 ft longitudinal distance). 

Four landings were outside of the “acceptable” performance criteria for longitudinal touchdown position - two baseline 
HDD in 1400 RVR (one as short 53 ft) with TDZ and CL and two SVS HDD in 1400 RVR without TDZ and CL lights 
(one as short as 26.8 ft). As mentioned before, one pilot landed 50 ft short of the threshold while flying the SVS HDD 
configuration in 1800 RVR with TDZ and CL lights. In all of these five cases, the pilots were essentially flying visually, 
without reference to HUD information or SVS/EFVS imagery. 

None of the concepts tested met the 10" 6 probability of occurrence auto-land criteria for T/D longitudinal position, lateral 
position from centerline, or sink rate. 
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Figure 4. Touchdown Footprint for Crew Display Concepts in 1400 RVR OTW and 150 ft DH 

4.2 Landing Performance Discussion 

Lateral deviation from centerline touchdown performance was excellent for all concepts tested. 

Expanding the EFVS visual segment from DH to the runway in visibilities as low as 1000 RVR appears to have had no 
negative consequences on approach/landing performance. First, and probably most important, no go-arounds were 
performed while using an EFVS. Second, approaches using the EFVS concept resulted in excellent longitudinal (mean= 
984 ft, a=381 ft) and lateral (mean= -0.49 ft, a=14.5 ft) touchdown performance regardless of OTW visibility level or if 
there were TDZ/CL lights or not. It is also interesting to note that the mean longitudinal distance from aim point was 
only 12 ft for this test when using the EFVS concept. Past NASA research 16 flying similar EFVS approach operations 
resulted in a mean longitudinal distance of 708 ft past the aim point. Applying recommendations from that experiment 
(automatic throttle retard to idle at 35 ft, providing pilots touchdown performance feedback during training and training 
each pilot to a common “desired” touchdown performance proficiency level) appears to have led to improvements in 
longitudinal T/D performance. 

When using the SVS HUD concept, longitudinal T/D performance and the ability to complete a landing were affected by 
the OTW visibility. As visibility decreased from 1400 ft to 1000 ft RVR, the number of go-arounds increased 
significantly - to where almost one-third of the approaches resulted in TOGA - and the mean distance from the 
touchdown aim point increased by 255 ft. 

Longitudinal T/D performance using the SVS HDD concept was influenced by the presence of TDZ/CL lights. The EPs 
were able to land within 24 feet (on average) of the aim point when the TDZ/CL lights were on but only within 1 19 ft 
(on average) when they were off. Also, as visibility decreased from 1800 to 1400, the number of go-arounds when using 
the SVS HDD concept increased by 13%. But in comparison to flying without SVS in these same conditions (i.e., flying 
the Baseline HDD in 1400 RVR), SVS provided a notable reduction in TOGAs (7 TOGAs vs. 4) likely by providing an 
aided search for the pilots (i.e., they knew where to look for the runway and may have had better awareness of the 
situation approaching the DH). 







The presence of TDZ/CL lights had a significant effect on longitudinal T/D position past threshold, but not on lateral 
deviation from centerline or T/D sink rate. With TDZ/CL lights, the mean longitudinal distance from threshold 
(collapsed across display concepts, mean=1046 ft) was much closer to the (1000 ft from threshold) aim point than when 
there were none (mean=803 ft). Also, in 1400 ft OTW RVR, pilots were almost twice as likely to complete a Baseline 
HDD approach to a landing if TDZ/CL lights were present. 

The T/D sink rate data shows significantly higher values across all conditions than expected. Since none of the display 
or lighting conditions seemingly affect the sink rate performance, the probable cause for this disparity, despite pre-test 
training, is the lack of simulator motion and the unfamiliar handling characteristics of a sidestick-controlled B-757. 

4.3 Workload 

Workload was assessed after each experimental run, independently for the PF and PNF, using the Air Force Flight 
Technical Center Workload Estimate Technique. 18 Workload ratings while using SEVS concepts were evaluated by 
conducting ANOVA analyses for the factors of display concept (Baseline HUD, SVS HUD, EFVS, Baseline HDD, and 
SVS HDD) and TDZ/CL lights (on, off) for the fixed 1400 RVR visibility/ 150 ft DH. Workload ratings while using 
SVS and EFVS were evaluated by conducting separate ANOVAs across OTW visibility and TDZ/CL lights. When 
necessary, Student-Newman-Keuls (SNK) post-hoc tests (a set at 0.05) were performed. 

4.3.1 SEVS Workload Ratings 

For the fixed 1400 OTW RVR/150 ft DH, ANOVAs revealed significant display concept differences (F(4,103)=3.39, 
p=0.012) for the PF workload ratings, but not for the PNF workload ratings (mean rating=2.7). Post-hoc tests showed 
two overlapping subsets for PF workload ratings: 1) EFVS (mean=2.9), SVS HUD (mean=3.1), Baseline HUD 
(mean=3.3) and 2) SVS HUD, Baseline HUD, SVS HDD (mean=3.6), Baseline HDD (mean=3.7). The PFs rated their 
workload as “being moderate, easily managed and having considerable spare time” when using the EFVS and that the 
EFVS required significantly less workload than when using the Baseline or SVS HDD concepts (rated as “being 
challenging but manageable and with adequate time available”). There were no appreciable workload differences 
between: 1) the three HUD concepts (EFVS, SVS, and Baseline) or 2) the Baseline and SVS HUD concepts and the 
Baseline and SVS HDD concepts. There were also no significant (p>0.05) differences among the TDZ/CL lights factor 
or the second-order interaction between display concept and TDZ/CL lights for either the PF or PNF workload ratings. 

4.3.2 EFVS Workload Ratings 

ANOVAs revealed no significant differences for OTW visibility, TDZ/CL lights, or the interaction between these two 
factors for the EFVS PF (mean=2.8) or PNF (mean=2.6) workload ratings. On average, both the PFs and PNFs rated 
their workload as being “moderate, easily managed and having considerable spare time” while using the EFVS display 
concept on approach through landing in visibilities as low as 1000 RVR. 

4.3.3 SVS Workload Ratings 

ANOVAs for the SVS HUD and SVS HDD concepts revealed no significant differences for OTW visibility, TDZ/CL 
lights, or the interaction between these two factors for the SVS HUD PF (mean=3.2) and PNF (mean=2.8) workload 
ratings or the SVS HDD PF (mean=3.4) and PNF (mean=2.6) workload ratings. On average, both the PFs and PNFs 
rated their workload as being “moderate, easily managed and having considerable spare time” while using the SVS 
display concepts on approach through landing in visibilities as low as 1000 RVR for the HUD and 1400 RVR for the 
HDD. 

4.4 Crew Display Concept Preferences 

The PF and PNF were independently asked to rank order the display concepts (Baseline HDD, SVS HDD, Baseline 
HUD, SVS HUD, and EFVS) from most preferred display (rank=l) to least preferred display (rank=5) for flying with in 
low-visibility conditions. The pilots ranked their low- visibility operations display concept preferences as follows: 1) 
EFVS {most preferred ), 2) SV HUD, 3) Baseline HUD, 4) SVS HDD, and 5) Baseline HDD {least preferred). 
Friedman’s test statistic showed there were significant differences among the displays for both the PF (x 2 (4)=31.145, 
p<0.001) and PNF (x 2 (4)=41.818, p<0.001) preference rankings. 



5. CONCLUSIONS 


A fixed-base pilot- in-the-loop simulation test was conducted at NASA Langley Research Center which evaluated the use 
of synthetic vision systems/enhanced flight vision systems (SVS/EFVS) in Next Generation Air Transportation System 
low visibility approach/landing operations at Chicago O’ Hare environment. Various scenarios tested the potential for 
EFVS for operations in visibility as low as 1000 ft runway visibility range (RVR) and SVS to enable lower decision 
heights or visibilities than can currently be flown today. 

Objective results indicate that expanding the EFVS visual segment from decision height (DH) to the runway in 
visibilities as low as 1000 ft RVR appears to be viable as longitudinal and lateral touchdown performance were 
excellent. Perhaps more important than the landing performance results is that the go-around rate was 0% when flying 
the EFVS concept, regardless of the out-the-window (OTW) visibility level (1000 or 1400 ft RVR) or if there were 
touchdown zone/center line (TDZ/CL) lights present or not. The enhanced flight visibility was held at approximately 
2400 ft. Subjective results also supported the EFVS visual segment expansion to the runway. This concept was rated as 
having less workload and was ranked as the crew’s preferred display concept (over the Baseline and SVS concepts 
tested) to fly with in low-visibility conditions. 

Objective and subjective results also lend support for using SVS (head-up) to enable lower decision heights and/or lower 
OTW visibility than are currently flown in today’s National Airspace System. Regardless of OTW visibility level or the 
absence/presence of TDZ/CL lights, all SVS concepts evaluated easily met the desired lateral touchdown criteria defined 
for this test. However, OTW visibility effects were found for go-around rate with the SVS HUD and SVS HDD 
concepts and longitudinal touchdown performance with the SVS HUD concept. SVS HUD operations in visibilities as 
low as 1400 RVR appear possible, but operations in 1000 ft RVR appear to require less intrusive implementations of the 
SVS imagery on the HUD to enable easier acquisition of the required visual landing cues at DH. SVS HDD operations 
in 1800 ft RVR with a 150 ft DH appear promising if TDZ/CL lights are present. Further study, including analysis of 
eye gaze data obtained in this experiment, is needed before conclusions can be drawn. 

In general, having TDZ/CL lights appears to have aided the pilots in landing closer to the touchdown aim point (1000 ft 
past the runway threshold). 

Future research should include motion-based simulation testing for the SVS HUD and HDD concepts to assess its impact 
on approach and landing performance, especially in sink rate control on touchdown. 
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